Skip to content

gh-116631: Fix race condition in test_shutdown_immediate_put_join#116670

Merged
colesbury merged 2 commits intopython:mainfrom
colesbury:gh-116631-put-join
Mar 13, 2024
Merged

gh-116631: Fix race condition in test_shutdown_immediate_put_join#116670
colesbury merged 2 commits intopython:mainfrom
colesbury:gh-116631-put-join

Conversation

@colesbury
Copy link
Copy Markdown
Contributor

@colesbury colesbury commented Mar 12, 2024

The test case had a race condition: if q.task_done() was executed after shutdown(immediate=True), then it would raise an exception because the shutdown call already emptied the queue. This happened rarely with the GIL (due to the switching interval), but frequently in the free-threaded build with the GIL disabled.

…oin`

The test case had a race condition: if `q.task_done()` was executed
after `shutdown(immediate=True)`, then it would raise an exception
because the immediate shutdown already emptied the queue. This happened
rarely with the GIL (due to the switching interval), but frequently in
the free-threaded build.
@colesbury
Copy link
Copy Markdown
Contributor Author

@EpicWink - I wasn't able to add you as a reviewer, but if you have time, I'd appreciate your feedback on the PR as well.

Copy link
Copy Markdown
Member

@gvanrossum gvanrossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand this test well enough to approve the change -- we ought to wait for @EpicWink, since it's his code, but sometimes it takes a week to get his input (he's just a volunteer). If this race condition is causing you grief, go ahead and merge it. I have one naming nit.

Comment thread Lib/test/test_queue.py Outdated
Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
@rhettinger rhettinger removed their request for review March 12, 2024 22:36
@EpicWink
Copy link
Copy Markdown
Contributor

EpicWink commented Mar 12, 2024

@YvesDup wrote the tests, with minimal fixing by me before the PR was merged.

I've found adding delays in the main thread usually surfaces the race conditions. I'll test when I'm on PC

Copy link
Copy Markdown
Contributor

@EpicWink EpicWink left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests succeed even with random delays spotted around

Comment thread Lib/test/test_queue.py
result = q.get()
self.assertEqual(result, "Y")
q.task_done()

This comment was marked as resolved.

Copy link
Copy Markdown
Member

@gvanrossum gvanrossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @EpicWink for the review. @colesbury Go ahead and merge! (No need to credit me for that variable rename. :-)

@colesbury colesbury merged commit 98ab21c into python:main Mar 13, 2024
@colesbury colesbury deleted the gh-116631-put-join branch March 13, 2024 18:56
@colesbury
Copy link
Copy Markdown
Contributor Author

@EpicWink, thanks for reviewing and testing this!

vstinner pushed a commit to vstinner/cpython that referenced this pull request Mar 20, 2024
…oin` (python#116670)

The test case had a race condition: if `q.task_done()` was executed
after `shutdown(immediate=True)`, then it would raise an exception
because the immediate shutdown already emptied the queue. This happened
rarely with the GIL (due to the switching interval), but frequently in
the free-threaded build.
adorilson pushed a commit to adorilson/cpython that referenced this pull request Mar 25, 2024
…oin` (python#116670)

The test case had a race condition: if `q.task_done()` was executed
after `shutdown(immediate=True)`, then it would raise an exception
because the immediate shutdown already emptied the queue. This happened
rarely with the GIL (due to the switching interval), but frequently in
the free-threaded build.
diegorusso pushed a commit to diegorusso/cpython that referenced this pull request Apr 17, 2024
…oin` (python#116670)

The test case had a race condition: if `q.task_done()` was executed
after `shutdown(immediate=True)`, then it would raise an exception
because the immediate shutdown already emptied the queue. This happened
rarely with the GIL (due to the switching interval), but frequently in
the free-threaded build.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip news tests Tests in the Lib/test dir

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants